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Status of Claims 

Claims 1-94 have been finally rejected and are the subject of this appeal. No 
other claims are pending. 

Status of Amendments 

All amendments submitted have been entered. 

Summary of Invention 

Generally, this invention relates to virtual reality environments and systems 
configured to implement such environments. A computer model that represents a virtual 
environment is modified by input from various participants. The virtual environment is 
displayed to the participants using sensory displays such as head-mounted visual and 
auditory displays which travel with the wearer and track the position and orientation of 
the wearer's head in space. Participants can look at each other within the virtual 
environment and see virtual body images of the other participants in a manner similar to 
the way that people in a physical environment see each other. Each participant can also 
look at his or her own virtual body in the same manner that a person in a physical 
environment can look at his or her own real body. The participants may work on a 
common task together and view the results of each other's actions. The system may 
represent the participants and other objects in the virtual environment with cursors. The 
cursors may each include pluralities of nodes configured in point hierarchies that allow 
efficient management and manipulation of the nodes. 

The Prior Art 

"Virtual Environment Display System" by S.S. Fisher, et al, ACM 1986 
Workshop on Interactive 3D Graphics, October 23-24, Chapel Hill, North Carolina 
(hereafter referred to as "Fisher"). Fisher discloses a virtual environment system that can 
implement a three-dimensional cursor controlled by a user (see bottom of page 4 under 
heading "Display Interaction"). 



51SM0802/P3W5RI 



2 



U.S. Patent 4,884,219 to Jonathan D. Waldren is titled "Method and Apparatus 
for the Perception of Computer-Generated Imagery" (hereafter referred to as "Waldren"). 
Waldren discloses a virtual reality system that senses head movements and provides 
support for multiple users (see Col. 6, lines 1-6). 

Issues 

(1) Are original claims 1-30 forfeited unless Applicants secure the allowance of claims 
31-94? 

(2) Does the recapture rule bar newly added claims 31-94? 
Grouping of Claims 

Claims 1-30 form group (1) and were collectively rejected for failure to state an error 
warranting reissue. Solely for the purposes of resolving issue (1) on this appeal, claims 
1-30 form stand or fall together. 

Claims 31-94 form group (2) that were collectively rejected under the recapture rule (see 
issue (2) above). However, for reasons set forth below in the Argument section, 
Applicants contend that claims 31-94 do not stand or fall together, since each claim must 
be examined to determine its scope relative to the original claims, the cited art, and the 
claim amendments. 

Argument 

1. Applicants contend that claims 1-30, which have not been broadened, are 
allowable independent of whether claims 31-94 are allowed. 

Applicants filed the present reissue application asserting a number of errors, 
including an error based on having claimed less than the Applicants were entitled to 
claim, e.g., by failing to claim a computer program embodied on an electronic medium. 
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Applicants have attempted to correct this error by submitting new claims 36-94 in 
addition to resubmitting original claims 1-30. However, the Examiner has rejected 
original claims 1-30 stating "Claims 1-30 will not become allowable unless applicants 
amend the new independent claims so that they are commensurate in scope with all 
aspects of the appropriate previously allowed claims... Should applicants merely cancel 
their new claims or fail to present allowable claims which correct errors other than those 
of a typographical or clerical nature they will have failed to demonstrate an error meriting 
reissue and all claims will stand rejected." 

Applicants contend the Examiner's interpretation of the reissue statute is contrary 
to that of the Federal Circuit. In Clement, the reissue application included claims 1-18, 
which corresponded to claims 1-18 of the original patent, and claims 49-52, which are admittedly 
broader than original patent's claims. The Federal Circuit held that claims 49-52 had been 
impermissibly broadened in violation of the recapture rule. However, the Court 
nevertheless held that original claims 1-18 were allowable despite the Examiner's 
attempts to invalidate them for a defective declaration (i.e., for failure to properly state an 
error warranting reissue). The Court noted that "because under 35 U.S.C. § 252 (1994) 
the surrender of the '179 patent does not take effect until the reissue patent issues, 
'original claims 1-18 [not subject to the recapture rule] continue to exist with their normal 
presumption of validity,' unaffected by the examiner's rejection based on the allegedly 
defective declaration." 1 Thus, Applicants contend that unless the Examiner is able to 
find a substantive reason for rejecting claims 1-30 (e.g., new prior art), claims 1-30 
should be allowed. 

2.1 The recapture rule was improperly applied to reject claims 31-94 despite the 
incorporation of all limitations germane to the prior art rejection and the 
incorporation of additional new features and limitations. 

Applicants filed the present reissue application within the two year statutory 
period for broadening reissue applications. The Examiner rejected claims 31-94 under 35 

1 In Re Clement, 45 USPQld 1161, 1167 (Fed. Or. 1997) 

5181-10SO2/P3O45RI 

4 



U.S.C. § 251 as being drawn to subject matter voluntarily given up by the Applicants in 
order to obtain a patent. Applicants respectfully traverse this rejection for at least the 
reasons set forth below. 

In applying the recapture rule, one must first determine in what aspect the reissue 
claims are broader than the patent claims, and then whether the broader aspects of the 
reissue claims relate to surrendered subject matter. 2 "To determine whether an applicant 
surrendered particular subject matter, we look to the prosecution history for arguments 
and changes to the claims made in an effort to overcome a prior art rejection. [Emphasis 
in original]" With regard to claim amendments, the recapture rule does not apply in the 
absence of evidence that the amendment was an admission that the scope of the claim 
was not patentable. 4 

Applicants contend that the only areas in which new claims 31-94 are broader 
than the original allowed claims 1-30 are areas that are not germane to surrendered 
subject matter. In particular, Applicants note that only one amendment in response to 
prior art was filed in the prior application 5 . That amendment was filed on March 8, 1996. 
As stated on page 15 of the amendment, Applicants specifically argued that independent 
claims 1, 26, and 30 were allowable because: 

"Waldren does not reach or suggest the positively recited features in Claim 1 of 
the first body emulating means including a first point hierarchy and a first data 
flow network, the first data point hierarchy for controlling a shape and an 
orientation of the first cursor. . .; [and] the second body emulating means 
including a second point hierarchy and a second data flow network, the second 
data point hierarchy for controlling a shape and an orientation of the second 
cursor..." [Emphasis added] 



2 See In Re Clement, 45 USPQ.ld 1161, 1164 (CAFC 1997). 

3 Hester Indus. Inc. v. Stein Inc., 46 USPQ.2d 1641, 1647 (CAFC 1998), citing In re Clement, 45 USPQ.2d 
1161, 1164 (CAFC 1997). 

4 See Hester at 1648, citing Clement at 1164. 

5 Applicants note that in the Office Action dated November 8, 1995 that preceded the amendment in the 
original case, the Examiner rejected pending claims 1-26. Claim 1 was rejected under 35 U.S.C. § 103 as 
obvious over Waldren (U.S. Patent No. 4,884,219) in light of Fisher, et al. (Virtual Environment Display 
System). 
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Applicants note that the above features are recited in new claims independent 
claims 31, 66, 72, 77, 90, and 94. In particular, Applicant notes that claims 31, 66, 72, 
77, 90, and 94 recite "the first cursor comprises a first plurality of nodes configured as a 
first point hierarchy" and "changing one or more attributes of a first cursor." 
Similarly, claims 31, 66, 72, 77, 90, and 94 recite "the second cursor comprises a second 
plurality of nodes configured as a second point hierarchy" and "changing one or more 
attributes of a second cursor." Applicants contend that these highlighted features are 
the only features germane to the rejection faced by Applicants in the original application. 
As such, Applicants contend that the remaining features of claims 31 and 66 that are 
broader than claim 1 are not subject to the recapture rule and are proper since the reissue 
application was filed within the two year period specified for broadening reissue 
applications under 35 U.S.C. § 251. The same reasoning applies to claims 32-65, 67-71, 
73-76, 78-89, and 91-93 which depend from claims 31, 66, 72, 77, and 90, respectively. 

Regarding the recapture rule, the Federal Circuit has stated that to determine 
whether an applicant surrendered particular subject matter, one looks to the prosecution 
history for arguments and changes to the claims made in an effort to overcome a prior art 
rejection. 6 While Applicants do not contend that only argued amendments are germane 
to prior art rejections, Applicants do maintain that arguments made during prosecution 
may be highly relevant in determining what is germane. 7 

The courts also look to the applicant's intent, and the court may draw inferences 
from changes in claim scope when other reliable evidence of the applicant's intent is not 

Q 

available. Deliberately canceling or amending a claim in an effort to overcome a 
reference strongly suggests that the applicant admits that the scope of the claim before the 
cancellation of amendment is unpatentable, but it is not dispositive because other 
evidence in the prosecution history may indicate the contrary. 9 



6 Id. 

7 Hester Indus. Inc. v. Stein Inc., 46 USPQ.2d 1641, 1647 (Fed. Cir. 1998) ( "we look to the prosecution 
history for arguments and changes to the claims made in an effort to overcome a prior art rejection "). 

8 See In Re Clement, 45 USPQ.2d 1161, 1164 (Fed. Cir. 1997). 



518M0S02/P3O45RI 



6 



Applicants contend that, after looking at the new claims and the prosecution 
history of the case, including the cited art, the claim amendments, and the argument, that 
it is clear that the new claims do not recapture subject matter Applicants intended to 
forfeit. In particular, an examination of both Fisher and Waldren references clearly 
shows that neither reference discloses the use of point hierarchies for cursor control and 
changing attributes. Given the nature of Applicants' amendments and the accompanying 
argument in light of the cited art, Applicants contend that these features are the ones 
germane to the rejection. In contrast, the Examiner has stated that Applicants new claims 
should be commensurate in scope with "all aspects" of the previously allowed claims. 

The Examiner also objects to the Applicants' avoidance of means-plus-function 
language in the newly added claims. The Examiner contends that adding claims that do 
not use means plus function language is an attempt to recapture subject matter. 
Applicants contend that since the claims were not amended to use means plus function 
language in response to the cited prior art, the recapture rule cannot be used to require 
means plus function language in the new broader claims. Applicants contend that the 
recapture rule cannot be applied in such a blanket manner as to require all aspects of a 
claim to be present in new claims. Such an application effectively rewrites the reissue 
statute to state that reissue applications are only available for applications that have not 
had any claim amendments. By applying the recapture rule in such an inequitable 
manner, broadening reissue applications are effectively eliminated for any claims that 
have had any amendments. Applicants contend that this result defies the clear language 
and intent of the reissue statute, as interpreted by the courts, which have stated that the 
reissue statute and recapture rule should be applied in an equitable manner. 10 

Assuming arguendo that some or all of Applicants' new claims broaden aspects 
that are related to the prior art rejection, Applicants contend that the claims are 
nevertheless proper in a broadening reissue application because other features in the 
claims are recited with greater specificity. Applicants highlight the Federal Circuit's 



9 See In Re Clement, 45 USPQ.2d 1 161, 1164 (Fed. Cir. 1997). 

10 See Ball Corp. v. U.S., 221 USPQ 289, 296 (Fed. Cir. 1984). 



5181-108O2/P3O45RJ 



7 



decision in Ball Corp, v. U.S., 221 USPQ 289 (CAFC 1984), in which the Court held that 
the recapture rule was avoided because the reissue claims were sufficiently narrow 
despite the broadened aspects of the claims. 11 "The purpose of [the Ball] exception to the 
recapture rule is to allow the patentee to obtain through reissue a scope of protection to 
which he is rightfully entitled for such overlooked aspects." 12 

In Ball Corp. v. U.S., Ball Corporation ("Ball") sued the United Stated 
government (the "Government") for unauthorized use of an invention claimed in Re. 
29,296. On interlocutory appeal, the Government raised two issues: (1) that the error 
alleged by Ball was insufficient under 35 U.S.C. § 251 to support reissue; and (2) that 
Ball was estopped from securing claims in the reissue application directed to a certain 
feature (i.e., a single feed). 14 

In the original application, Ball had claims directed to an antenna of cylindrical 
configuration with a feed means including a single feed line with "at least one conductive 
lead" (claim 8 of the original application) and "a plurality of leads" (claim 9 of the 
original application). 15 The examiner rejected claims 1-8, and indicated that claims 9-10 
should be limited to a "plurality of feedlines". 16 Ball amended the claims, and in a 
second office action the examiner suggested the allowability of the plurality of feedlines 
claims if presented in independent form. 17 The remaining claims were rejected over 
newly cited art, a U.S. Patent No. 2,234,234 to Cork, et al., that discloses a single 

1 8 

feedline. In response Ball amended its claims to recite "a plurality of leads," and 
canceled the original claims 7 and 8. 19 The patent issued, and within two years Ball 
applied for a broadening reissue. 20 



11 See Hester at 1649, citing Ball at 296. 

12 Hester at 1650. 

13 See Ball at 290. 

14 Id. at 293. 

15 See Id. at 291-292. 

16 See Id. 

17 See Id. 

18 See Id. 

19 See Ball at29I-292. 
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During the reissue proceedings, Ball pursued and received allowance of claims 
that recited a single feedline along with several new features (e.g., a dielectric material 
filling the cavity of the antenna). 21 These claims did not include the "cylindrical" 
limitation from the previously issued claims. The Government filed for summary 
judgment, arguing that Ball could not obtain claims to the single feedline because this 
feature had deliberately canceled claims to the single feedline in the original 
application. 23 The district court denied the Government's motion, and on appeal the 
Federal Circuit affirmed. 24 

The Federal Circuit stated that the proper focus was the "scope" of the claims, not 
on the individual features or elements purportedly given up during prosecution of the 
original application. 25 "We are aware of the principle that a claim that is broader in any 
respect is considered to be broader than the original claims even though it may be 
narrower in other respects. That rule will not bar Ball from securing the reissue claims 
here on appeal." 26 The Court also noted that the recapture rule should be applied 
equitably: "[t]he recapture rule. . .is based on equitable principles. The rigidity of the 
broader-in-any-respect rule makes it inappropriate in the estoppel situation presented in 
this appeal." 27 Thus, while the Court noted that the reissue claims were broader in certain 
respects and narrower in certain respects, the Court nevertheless held that the broadening 
of the claims was proper under 35 U.S.C. § 251 because the reissue application had been 
filed within the two year period specified in the statute. 

Applicants note that in new claims 31-94 a large number of features are recited 
that were overlooked in the original application. For example, claim 37 recited features 
not recited in the original claims including "an Ethernet link, a phone link, and ISDN 
line, or a satellite link." Similarly, claim 44 also recites features not recited in the 

21 See Id at 292-293. 

22 See Id. 

23 See Ball at 293. 

24 See Id. 

25 Id. at 295. 

26 Id. at 295-296. 

27 Id. at 296. 

28 See Id. 
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original claims such as "machines, articles of manufacture, animals, molecules..." Other 
examples include claim 68, which recites "one or more audio display devices configured 
to produce three-dimensional sounds. . and claim 69, which recites a number of 
different embodiments of the "body part sensing means". Thus, Applicants contend that 
under Ball, Clement, and Hester, claims 31-89 are properly presented in this broadening 
reissue application. 

Applicants must respectfully disagree with and traverse the Examiner's assertion 
that "the final language of the allowed claim is the determinant of what is germane." 
Under this reasoning, there could never be a broadening reissue application on any 
amended claims because everything in the amended claims would have to be copied into 
the reissue claims. This defies the clear language of the case law from the Federal Circuit 
noted above. As the cases cited above indicate, there is an in-depth analysis that must 
take place, including looking at both the cited art and the applicant's response. The 
Federal Circuit has stated that in a reissue case dealing with the recapture rule, the focus 
is not on the specific limitations or on the elements of the claims, but rather on the scope 
of the claims and the intent of the applicant. 

Regarding the Examiner's contention that using non-means plus function is an 
attempt at recapture, Applicants note that the patent issued on December 24, 1996, well 
before the Federal Circuit's decision in Chiuminatta Concrete Concepts, Inc. v. Cardinal 
Industries, Inc., 145 F.3d 1303 (Fed. Cir. 1998), which severely limited the scope of 
equivalents available to means-plus-function claims under 35 U.S.C. 112. Thus, 
Applicant's election to pursue non-means-plus-function claims does not per se result in 
broader claims when compared with the scope of the claims at the time the patent 
issued. 

2.2 Claims 31-65, 66-71, 90-93, and 94 are materially different from original claims 
1-30 and as such are not subject to the recapture rule. 

29 See Ball Corp. v. United States, 221 USPQ 289, 296 (Fed. Cir. 1984). 
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Furthermore, Applicants contend that the new claims are materially different from 
the claims in the original patent. As noted in Chisum on Patents, a patentee may obtain 
on reissue a claim that varies materially from the claim originally surrendered even 
though it omits a limitation intentionally added to obtain issuance of the patent. 30 

Applicants contend that new claims 31-96 recite additional features that render 
the new claims materially different from claims 1-30. For example, claim 36 recites 
"wherein the second set of data is received across a network," and claim 37 recites "an 
Ethernet link, a phone link, and ISDN line, or a satellite link." These features are not 
expressly recited in the original claims. Similarly, claim 44 also recites features not 
recited in the original claims such as "machines, articles of manufacture, animals, 
molecules..." Other examples include claim 68, which recites "one or more audio display 
devices configured to produce three-dimensional sounds..." and claim 69, which recites 
a number of different embodiments of the "body part sensing means". 

Applicants also note that software on medium claims such as claims 31-65, and 
94, and kit claims such as claims 66-71 and 90-93 are materially different than the 
apparatus and method claims 1-30. Claims 31-65 and 94 are directed to a computer 
software program embodied on a computer-readable medium. Claims 31-65 are directed 
to a computer program that processes data, emulate bodies, and maintains a database. 
The data represents the physical status of a part of a body, and this information is used to 
position cursors in a virtual environment, which is integrated into the database. 

Claim 94 is directed to a computer software program embodied on a computer- 
readable medium that is configured, in part, to "modify a virtual three-dimensional work 
piece based on the motion of the first cursor within the virtual world". Applicants 
contend that the invention to which this claim is directed has a different mode of 
operation (i.e., operating on a virtual work piece), a different function (i.e., allowing 
operations to be performed on a virtual work piece), and a different effect (i.e., a 

30 See Donald S. Chisum, Chisum on Patents §J5.03[2][e] (emphasis added). 
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modified work piece). Applicants note that original claims 1-30 do not expressly recite a 
"work piece". Applicants also note that none of original claims 1-30 are expressly 
directed to a computer software program embodied on a computer-readable medium. 
Similarly, claims 66-71 and 90-93 are directed at a kit for creating an interactive, multi- 
user three-dimensional virtual reality world. These claims also recite materially different 
subject matter from original claims 1-30. For example, claims 90-93 recite a kit having 
computer software that is configured to "modify a virtual three-dimensional work piece 
based on the motion of the first cursor within the virtual world." Claims 66-71 recite a 
kit for creating a virtual reality world including "two or more body part sensing means" 
and computer software configured to "process... data from [the] body part sensors, 
wherein the data represents the physical status of a first body part relative to a first 
reference point..." These features are materially different from those recited in original 
claims 1-30. Thus, for the reasons set forth above, Applicants contend that claims 31-94 
are not subject to the recapture rule. 

Conclusion 

Applicants contend that the recapture rule does not act as a complete bar to all 
claims 1-94 as presented herein. Thus, Applicants respectfully request that the Board of 
Appeals reverse the Examiner's rejections of claims 1-94 and remand the case for 
examination. 



Respectfully submitted, 




Dan R. Christen 
Reg. No. 39,943 

ATTORNEY FOR APPLICANT(S) 



Conley, Rose & Tayon, P.C. 
P.O. Box 398 
Austin, TX 78767-0398 
Phone: (512) 476-1400 



Date: flS£> Z/ ; ZOO ( 



5181-10SO2/P3O45RI 



12 




Appendix - Claims 



1 1. (Amended) A simulating apparatus comprising: 
2 

3 modeling means for creating a model of a physical environment in a computer database; 

4 first body sensing means, disposed in close proximity to a part of a first body, for sensing 

5 a physical status of the first body part relative to a first reference position; 

6 second body sensing means, disposed in close proximity to a part of a second body, for 

7 sensing a physical status of the second body part relative to a second reference 

8 position; 

9 first body emulating means, coupled to the first body sensing means, for creating a first 

10 cursor in the computer database, the first cursor including plural first cursor nodes 

11 and emulating the physical status of the first body part, the first body emulating 

12 means including a first point hierarchy and a first data flow network, the first 

13 point hierarchy for controlling a shape and an orientation of the first cursor and 

14 for attaching each of the plural first cursor nodes hierarchically with at least one 

15 other of the plural first cursor nodes, the first data flow network for controlling 

16 motion of the first cursor and the first data flow network including a first 

17 interconnection of first input units, first function units and first output units, the 

18 first input units receiving the physical status of the first body part, each first 

19 function unit including at least one input and at least one output and calculating, 

20 based on the at least one input, a value for each of the at least one output, and the 
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21 first output units for producing position and orientation values for a portion of the 

22 plural first cursor nodes; 

23 first integrating means, coupled to the modeling means and to the first emulating means, 

24 for integrating the first cursor with the model; 

25 second body emulating means, coupled to the second body sensing means, for creating a 

26 second cursor in the computer database, the second cursor including plural second 

27 cursor nodes and emulating the physical status of the second body part, the second 

28 body emulating means including a second point hierarchy and a second data flow 

29 network, the second point hierarchy for controlling a shape and an orientation of 

30 the second cursor and for attaching each of the plural second cursor nodes 

31 hierarchically with at least one other of the plural second cursor nodes, the second 

32 data flow network for controlling motion of the second cursor and the second data 

33 flow network including a second interconnection of second input units, second 

34 function units and second output units, the second input units receiving the 

35 physical status of the second body part, each second function unit including at 

36 least one input and at least one output and calculating, based on the at least one 

37 input, a value for each of the at least one output, and the second output units for 

38 producing position and orientation values for a portion of the plural second cursor 

39 nodes; and 

40 second integration means, coupled to the modeling means and to the second body 

41 emulating means, for integrating the second cursor with the model. 
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1 2. The apparatus according to claim 1 further comprising first model display means for 

2 displaying a view of the model. 

1 3. The apparatus according to claim 2 wherein the first model display means includes 

2 view changing means for changing the view of the model in response to a change in the 

3 physical status of the second cursor in the model. 

1 4. The apparatus according to claim 3 wherein the second cursor includes a first optical 

2 axis which moves together therewith, and wherein the view of the model produced by the 

3 first model display means corresponds to the view taken along the first optical axis. 

1 5. The apparatus according to claim 4 wherein the first model display means displays the 

2 first cursor together with the model when the first optical axis faces the location of the 

3 first cursor. 

1 6. The apparatus according to claim 5 wherein the first cursor depicts the first body part 

2 being emulated. 

1 7. The apparatus according to claim 1 wherein the model includes a virtual object, and 

2 further comprising first object manipulating means, coupled to the first body emulating 

3 means, for manipulating the virtual object with the first cursor in accordance with 

4 corresponding gestures of the first body part. 
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1 8. The apparatus according to claim 7 further comprising second object manipulating 

2 means, coupled to the second body emulating means, for manipulating the virtual object 

3 with the second cursor in accordance with corresponding gestures of the second body 

4 part. 

1 9. The apparatus according to claim 8 further comprising first model display means for 

2 displaying a view of the model. 

1 10. The apparatus according to claim 9 wherein the first model display means includes 

2 view changing means for changing the view of the model in response to a change in the 

3 physical status of the second cursor in the model. 

1 11. The apparatus according to claim 10 wherein the second cursor includes an optical 

2 axis which moves together therewith, and wherein the view of the model corresponds to 

3 the view taken along the optical axis. 

1 12. The apparatus according to claim 1 1 wherein the first model display means displays 

2 the first cursor together with the model when the optical axis faces the location of the first 

3 cursor. 

1 13. The apparatus according to claim 12 wherein the first cursor depicts the first body 

2 part being emulated. 
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1 14. The apparatus according to claim 13 wherein the first model display means displays 

2 the second cursor together with the model when the optical axis faces the location of the 

3 second cursor. 

1 15. The apparatus according to claim 14 wherein the second cursor depicts the second 

2 body part being emulated. 
1 

1 16. The apparatus according to claim 15 further comprising second model display means 

2 for displaying a view of the model, the view of the model changing in response to the 

3 physical status of the first cursor in the model. 
1 

1 17. The apparatus according to claim 16 wherein the first cursor includes a second 

2 optical axis which moves together therewith, and wherein the view of the model 

3 produced by the second model display means corresponds to the view taken along the 

4 second optical axis. 

1 18. The apparatus according to claim 17 wherein the second model display means 

2 displays the second cursor together with the model when the second optical axis faces the 

3 location of the second cursor. 

1 19. The apparatus according to claim 18 wherein the first body part is a part of a body of 

2 a first human being. 
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1 20. The apparatus according to claim 19 wherein the first model display means 

2 comprises a first head-mounted display. 

1 21. The apparatus according to claim 20 wherein the first head-mounted display 

2 comprises: 

3 a first display for displaying the model to a first eye; and 

4 a second display for displaying the model to a second eye. 

1 22. The apparatus according to claim 1 wherein the first and second displays together 

2 produce a stereophonic image. 

1 23. The apparatus according to claim 21 wherein the first head-mounted display further 

2 comprises: 

3 a first audio display for displaying a sound model to a first ear; and 

4 a second audio display for displaying the sound model to a second ear. 

1 24. The apparatus according to claim 21 wherein the first and second displays display 

2 the model as a series of image frames, and wherein the model display means further 

3 comprises frame synchronization means, coupled to the first and second displays, for 

4 synchronizing the display of the series of frames to the first and second displays. 
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1 25. The apparatus according to claim 19 wherein the second body part is a part of a 

2 body of a second human being. 



1 26. A simulating apparatus comprising: 

2 a modeling means for creating a virtual world model of a physical environment in a 

3 computer database; 

4 a first sensor for sensing a first real world parameter; 

5 first emulating means, coupled to the first sensor for emulating a first virtual world 

6 phenomenon in the virtual world model, the first emulating means including a first 

7 point hierarchy and a first data flow network, the first point hierarchy for controlling 

8 a shape and an orientation of a first cursor, including plural first cursor nodes, and for 

9 attaching each of the plural first cursor nodes hierarchically with at least one other of 

10 the plural first cursor nodes, the first data flow network for controlling motion of the 

1 1 first cursor and the first data flow network including a first interconnection of first 

12 input units, first function units and first output units, the first input units receiving the 

13 physical status of the first body part, each first function unit including at least one 

14 input and at least one output and calculating, based on the at least one input, a value 

15 for each of the at least one output, and the first output units for producing position 

16 and orientation values for a portion of the plural first cursor nodes; 

17 a second sensor for sensing a second real world parameter; and 
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18 second emulating means, coupled to the second sensor, for emulating a second virtual 

19 world phenomenon in the virtual world model, the second emulating means including 

20 a second point hierarchy and a second data flow network, the second point hierarchy 

21 for controlling a shape and an orientation of a second cursor, including plural second 

22 cursor nodes, and for attaching each of the plural second cursor nodes hierarchically 

23 with at least one other of the plural second cursor nodes, the second data flow 

24 network for controlling motion of the second cursor and the second data flow network 

25 including a second interconnection of second input units, second function units and 

26 second output units, the second input units receiving the physical status of the second 

27 body part, each second function unit including at least one input and at least one 

28 output and calculating, based on the at least one input, a value for each of the at least 

29 one output, and the second output units for producing position and orientation values 

30 for a portion of the plural second cursor nodes. 

1 27. An apparatus according to claim 21, wherein the first body sensing means includes a 

2 facial expression sensor using conductive ink. 

1 28. An apparatus according to claim 1, wherein the first body sensing means includes a 

2 facial expression sensor including a strain gauge. 

1 29. An apparatus according to claim 1, wherein the first body sensing means includes a 

2 pneumatic input device. 
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1 30. A simulating method, comprising the steps of: 

2 creating a virtual environment; 

3 constructing virtual objects within the virtual environment using a point hierarchy and 

4 a data flow network for controlling motion of nodes of the virtual objects wherein the 

5 step of constructing includes 

6 attaching each node of the virtual objects hierarchically with at least one other 

7 of the nodes to form the point hierarchy, each of the nodes of the virtual 

8 objects having a position and an orientation, and 

9 building the data flow network as an interconnection of input units, function 

10 units and output units, wherein said input units receive data from sensors and 

11 output the received data to at least one of said function units, wherein each of 

12 said function units includes at least one input and at least one output, each 

13 function unit generating a value for the at least one output based on at least 

14 one of data received from at least one of the input units and data received 

15 from an output of at least one other of said function units, and wherein the 

16 output units generate the position and the orientation of a portion of the nodes 

17 of the virtual objects; 

18 inputting data from sensors worn on bodies of at least two users; 

19 converting the inputted data to position and orientation data; 
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20 modifying by using the data flow network, the position and the orientation of the 

21 nodes of the virtual objects based on the position and orientation data; 

22 determining view points of said at least two users; 

23 receiving a synchronization signal; 

24 calculating image frames for each eye of each of said at least two users; 

25 displaying the image frames to each of said eyes of said at least two users; 

26 obtaining updated position and orientation values of said at least two users; 

27 determining if the virtual environment has been modified; 

28 redefining positions and orientations of the nodes of the virtual object if the virtual 

29 environment has been modified; 

30 recalculating the image frames for each of said eyes of said at least two users; and 

31 displaying the recalculated image frames to each of said eyes of said at least two 

32 users. 

1 31. (Amended) A computer software program embodied on a computer-readable 

2 medium, wherein the software program comprises a plurality of instructions, wherein the 

3 plurality of instructions are configured to: 
4 

5 process a first set of data from a first body sensor, wherein the first set of data represents 

6 the physical status of a part of a first body relative to a first reference point; 
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7 

8 process a second set of data from a second body sensor, wherein the second set of data 

9 represents the physical status of a part of a second body relative to a second reference 
10 point; 

11 

12 emulate the first body in a virtual three-dimensional environment by changing one or 

13 more attributes of a first cursor, wherein the first cursor comprises a first plurality of 

14 nodes configured as a first point hierarchy; 
15 

16 emulate the second body in the virtual three-dimensional environment by changing one or 

17 more attributes of a second cursor, wherein the second cursor comprises a second 

18 plurality of nodes configured as a second point hierarchy; 
19 

20 position the first cursor and the second cursor within the virtual environment; and 
21 

22 integrate the first cursor and the second cursor and the virtual environment into a 

23 database. 

1 32. (Amended) The computer software program as recited in claim 31, wherein the 

2 plurality of instructions are further configured to move two or more of the nodes in the 

3 first plurality of nodes in response to the first set of data indicating that one or more 

4 points in the hierarchy moved. 
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1 32. The computer software program as recited in claim 31, wherein the plurality of 

2 instructions are further configured to move two or more of the nodes in response to the 

3 first set of data indicating that one point in the hierarchy moved. 

1 33. The computer software program as recited in claim 31, wherein the virtual 

2 environment comprises both visual objects and non-visual objects. 

1 34. The computer software program as recited in claim 33, wherein the non-visual 

2 objects comprise auditory objects. 

1 35. (Amended) The computer software program as recited in claim 31, wherein the first 

2 reference point and the second reference point are the same point. 

1 36. The computer software program as recited in claim 31, wherein the second set of 

2 data is received across a network. 

1 37. The computer software program as recited in claim 36, wherein the network 

2 comprises one of the following: an Ethernet link, a phone line link, an ISDN link, or a 

3 satellite link. 

1 38. (Amended) The computer software program as recited in claim 31, wherein the 

2 plurality of instructions are further configured to render the virtual environment in stereo. 
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1 39. The computer software program as recited in claim 31, wherein the plurality of 

2 instructions are further configured to create the virtual environment. 

1 40. The computer software program as recited in claim 31, wherein the plurality of 

2 instructions are further configured to load the virtual environment from a storage device. 

1 41. (Amended) The computer software program as recited in claim 31, wherein the 

2 plurality of instructions are further configured to store the virtual environment to a 

3 storage device. 

1 42. The computer software program as recited in claim 31, wherein the cursors depict 

2 non-humanoid structures. 

1 43. (Amended) The computer software program as recited in claim 31, wherein the first 

2 cursor depicts at least part of a human figure. 

1 44. The computer software program as recited in claim 31, wherein each cursor depicts 

2 an separate object selected from the group comprising: machines, articles of 

3 manufacture, animals, molecules, human figures, human body parts, tools, and three- 

4 dimensional objects. 
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1 45. The computer software program as recited in claim 31, wherein the first point 

2 hierarchy controls the shape and orientation of the first cursor, and wherein the second 

3 point hierarchy controls the shape and orientation of the second cursor. 

1 46. The computer software program as recited in claim 45, wherein the motion of the 

2 first cursor and the first cursor's plurality of nodes are governed by a data flow network. 

1 47. The computer software program as recited in claim 31, wherein said second body is 

2 remotely located. 

1 48. The computer software program as recited in claim 31, wherein at least part of said 

2 database is configured to be remotely located. 

1 49. The computer software program as recited in claim 31, wherein said plurality of 

2 instructions are configured to be executed by a central processor that sends image frames 

3 to said first body and said second body, wherein said first body and said second body are 

4 remotely located relative to each other. 

1 50. The computer software program as recited in claim 31, wherein the database is 

2 configured to be shared by multiple instances of said software program executing in 

3 remote locations connected by linking technology. 
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1 51. The computer software program as recited in claim 31, wherein the software program 

2 is configured to be executed in two or more remote locations simultaneously, wherein the 

3 two or more remote locations are coupled by linking technology. 

1 52. The computer software program as recited in claim 51, wherein the database is 

2 shared between the two or more locations. 

1 53. The computer software program as recited in claim 51, wherein the database is 

2 duplicated at each of the two or more locations. 

1 54. The computer software program as recited in claim 51, wherein the plurality of 

2 instructions is configured to compress communications between the two or more 

3 locations. 

1 55. The computer software program as recited in claim 51, wherein the plurality of 

2 instructions is configured to share images between the two or more locations. 

1 56. The computer software program as recited in claim 31, wherein the behavior of said 

2 cursors in said virtual environment is constrained by the laws of physics. 

1 57. The computer software program as recited in claim 31, wherein the plurality of 

2 instructions are configured to superimpose prerecorded behavior on the models. 
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1 58. The computer software program as recited in claim 31, wherein the plurality of 

2 instructions are configured to algorithmically derive at least part of the said first and 

3 second sets of data. 

1 59. (Amended) The computer software program as recited in claim 31, wherein the first 

2 and second sets of data do not vary in real-time. 

1 60. (Amended) The computer software program as recited in claim 3 1 , wherein the first 

2 and second sets of data vary in real-time. 

1 61. The computer software program as recited in claim 31, wherein the first set of data 

2 represents changes in the orientation of the part of the first body relative to the first 

3 reference point. 

1 62. The computer software program as recited in claim 31, wherein the first set of data 

2 represents changes in the position of the part of the first body relative to the first 

3 reference point. 

1 63. The computer software program as recited in claim 31, wherein the first set of data 

2 represents changes in the shape of the part of the first body relative to the first reference 

3 point. 
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64. The computer software program as recited in claim 31, wherein the plurality of 
instructions are further configured to generate three-dimensional sounds. 

65. The computer software program as recited in claim 31, wherein the first point 
hierarchy and the second point hierarchy are different part of a single point hierarchy. 

1 66. (Amended) A kit for creating an interactive, multi-user three-dimensional virtual 

2 reality world, the kit comprising: 

3 

4 two or more body part sensing means, each configured to be worn by a separate body; 

5 and 
6 

7 a computer software program embodied on a computer-readable media, the program 

8 comprising a plurality of instructions, wherein the instructions are configured to: 



9 



10 



process a first set of data from the first body part sensor, wherein the first set of 



11 



data represents the physical status of a first part of a first body relative to a 



12 



first reference point; 



13 



14 



process a second set of data from the second body part sensor, wherein the second 



15 



set of data represents the physical status of a second part of a second body 



16 



relative to a second reference point; 
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17 

18 emulate the first body in the three-dimensional virtual world by changing one or 

19 more attributes of a first cursor, wherein the first cursor comprises a first 

20 plurality of nodes configured as a first point hierarchy; 
21 

22 emulate the second body in the three-dimensional virtual world by changing one 

23 or more attributes of a second cursor, wherein the second cursor comprises a 

24 second plurality of nodes configured as a second point hierarchy; 
25 

26 position the first cursor and the second cursor within the virtual world; and 
27 

28 integrate the first cursor and the second cursor and the virtual world into a 

29 database. 

1 67. The kit as recited in claim 66, further comprising one or more display devices 

2 configured to display images of the virtual world. 

1 68. The kit as recited in claim 66, further comprising one or more audio display devices 

2 configured to produce three-dimensional sounds as part of the virtual world. 

1 69. The kit as recited in claim 66, wherein said first and second body part sensing means 

2 are selected from the group consisting of: eye tracking devices, cameras, clothing-based 

3 sensors, force feedback devices, ultrasonic tracking devices, voice recognition devices, 
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4 video tracking devices, keyboards, pneumatic input devices, facial expression sensors, 

5 magnetic tracking sensors, infrared tracking devices, and telemetry sensing devices. 

1 70. The kit as recited in claim 66, wherein said first cursor and said second cursor 

2 represent objects selected from the group comprising: machines, articles of manufacture, 

3 animals, molecules, human figures, human body parts, tools, and three-dimensional 

4 objects. 

1 71. The kit as recited in claim 66, further comprising one or more computer systems 

2 configured to: 

3 execute said computer software program; 

4 receive said first set of data and said second set of data; and 

5 generate an image of the virtual world for output. 

1 72. (Amended) A computer system configured to creating an interactive, multi-user 

2 three-dimensional virtual reality world, the computer system comprising: 

3 a central processing unit; 

4 a memory coupled to the central processing unit; 

5 one or more display processors; and 

6 a computer software program embodied on a computer-readable media, the program 

7 comprising a plurality of instructions, wherein the instructions are configured to: 
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8 process a first set of data from a first body part sensor, wherein the first set of data 

9 represents the physical status of a first part of a first body relative to a first 

10 reference point; 

1 1 process a second set of data from a second body part sensor, wherein the second 

12 set of data represents the physical status of a second part of a second body 

13 relative to a second reference point; 

14 emulate the first body in the three-dimensional virtual world by changing one or 

15 more attributes of a first cursor, wherein the first cursor comprises a first 

16 plurality of nodes configured as a first point hierarchy; 

17 emulate the second body in the three-dimensional virtual world by changing one 

18 or more attributes of a second cursor, wherein the second cursor comprises a 

19 second plurality of nodes configured as a second point hierarchy; 

20 position the first cursor and the second cursor within the virtual world; and 

21 integrate the first cursor and the second cursor and the virtual world into a 

22 database. 



73. The computer system as recited in claim 72, further comprising two or more body 
part sensors configured to be coupled to said central processing unit, wherein said body 
part sensors are selected from the group consisting of: eye tracking devices, cameras, 
clothing-based sensors, force feedback devices, ultrasonic tracking devices, voice 
recognition devices, video tracking devices, keyboards, pneumatic input devices, facial 
expression sensors, magnetic tracking devices, infrared tracking devices, and telemetry 
sensing devices. 
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74. The computer system as recited in claim 72, further comprising two or more display 
devices configured to be coupled to said central processing unit and configured to display 
images of the virtual world generated by said central processing unit. 

75. The computer system as recited in claim 72, further comprising two or more audio 
display devices configured to be coupled to said central processing unit and configured to 
reproduce sound from the virtual world generated by the central processing unit. 

1 76. (Amended) The computer system as recited in claim 70, wherein said first cursor 

2 and said second cursor represent objects selected from the group comprising: machines, 

3 articles of manufacture, animals, molecules, human figures, human body parts, tools, and 

4 three-dimensional objects. 

77. A method for interacting with a virtual world comprising: 

processing a first set of data from a first sensor, wherein the first set of data represents the 
physical status of a part of a first body relative to a first reference point; 

processing a second set of data from a second sensor, wherein the second set of data 
represents the physical status of a part of a second body relative to a second reference 
point; 
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10 emulating the first body in the virtual world by changing one or more attributes of a first 

1 1 cursor, wherein the first cursor comprises a first plurality of nodes configured as a 

12 first point hierarchy; 
13 

14 emulate the second body in the virtual world by changing one or more attributes of a 

15 second cursor, wherein the second cursor comprises a second plurality of nodes 

16 configured as a second point hierarchy; 
17 

18 calculating the position of the first cursor and the second cursor within the virtual world; 

19 and 
20 

21 integrating the first cursor and the second cursor into a database representing the virtual 

22 world. 

1 78. (Amended) The method as recited in claim 77, further comprising generating 

2 stereophonic three-dimensional sounds to produce an experience that a source for the 

3 sounds is located in a specific location in the virtual world. 

1 79. The method as recited in claim 77, further comprising communicating said second 

2 set of data across a network link. 
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1 80. The method as recited in claim 79, wherein said network link is selected from the 

2 group comprising: an Ethernet link, a phone line link, an ISDN link, or a satellite link. 

1 81. The method as recited in claim 77, further comprising communicating changes in 

2 said database across a network. 

1 82. The method as recited in claim 77, further comprising algorithmically deriving the 

2 second set of data. 

1 83. The method as recited in claim 77, further comprising: 

2 generating at least a partial first image of the virtual world for said first body; and 

3 generating at least a partial second image of the virtual world for said second body. 

1 84. (Amended) The method as recited in claim 83, wherein said first partial image is 

2 generated from a viewpoint related to the position and orientation of said first cursor in 

3 said virtual world. 

1 85. The method as recited in claim 84, further comprising communicating said first 

2 partial image across a network. 
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1 86. The method as recited in claim 84, wherein said second partial image is generated 

2 from a viewpoint related to the position and orientation of said second cursor in said 

3 virtual world. 

1 87. The method as recited in claim 77, wherein said physical status comprises 

2 information selected from the following group: temperature, blood pressure, heart rate, 

3 radiation, position, and orientation. 

1 88. The method as recited in claim 77, further comprising updating the database to 

2 change objects in the virtual world in response to said first set of data and said second set 

3 of data. 

1 89. (Amended) The method as recited in claim 77, wherein said first cursor and said 

2 second cursor represent objects selected from the group comprising: machines, articles of 

3 manufacture, animals, molecules, human figures, human body parts, tools, and three- 

4 dimensional objects. 

1 90. A kit for creating virtual three-dimensional objects in an interactive, multi-user three- 

2 dimensional virtual reality world, the kit comprising: 

3 one or more body part sensing means configured to sense a first user body; 

4 a display device configured to display a first image; and 

5 a computer software program embodied on a computer-readable media, the program 

6 comprising a plurality of instructions, wherein the computer software program is 
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7 configured to be executed on a computer coupled to said one or more body sensing 

8 means and said display device, wherein the instructions are configured to: 

9 receive a first set of data from the first body part sensing means; 

10 emulate the first body in the three-dimensional virtual world by changing one or 

1 1 more attributes of a first cursor, wherein the first cursor comprises a first 

12 plurality of nodes configured as a first point hierarchy; 

13 move the first cursor within the virtual world based on the first set of data; 

14 modify a virtual three-dimensional work piece based on the motion of the first 

15 cursor within the virtual world; and 

16 update a database to reflect the changes to the virtual three-dimensional work 

17 piece. 

1 91. The kit as recited in claim 90, wherein the first cursor is a virtual tool, and wherein 

2 the three-dimensional work piece is a virtual sculpture. 

1 92. The kit as recited in claim 90, wherein the instructions of said computer software 

2 program are further configured to: 

3 receive a second set of data from one or more second body sensing means 

4 configured to sense a second user body; 

5 emulate the second body in the three-dimensional virtual world by changing one 

6 or more attributes of a second cursor, wherein the second cursor comprises a second 

7 plurality of nodes configured as a second point hierarchy; 
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8 



move the second cursor within the virtual world based on the second set of data; 



9 



and 



10 



modify the virtual three-dimensional work piece based on the motion of the 



11 



second cursor within the virtual world. 



1 93. The kit as recited in claim 92, wherein the instructions of the computer software 

2 program are configured to cause the updated database to be rendered on said first display 

3 device and on a second display device configured to display a second image, wherein 

4 said first image is rendered based on a first viewpoint corresponding to said first cursor's 

5 position in said virtual three-dimensional world, and wherein said second image is 

6 rendered based on a second viewpoint corresponding to said second cursor's position in. 

7 said virtual three-dimensional world. 

1 94. (Amended) A computer software program embodied on a computer-readable media, 

2 the program comprising a plurality of instructions, wherein the computer software 

3 program is configured to be executed on a computer coupled to one or more body sensing 

4 means and one or more display devices, wherein the instructions are configured to: 

5 receive a first set of data from the body sensing means; 

6 emulate a first body in a three-dimensional virtual world by changing one or more 

7 attributes of a first cursor, wherein the first cursor comprises a first plurality of 

8 nodes configured as a first point hierarchy; 

9 move the first cursor within the virtual world based on the first set of data; 
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10 modify a virtual three-dimensional work piece based on the motion of the first 

1 1 cursor within the virtual world; 

12 update a database to reflect the changes to the virtual three-dimensional work 

13 piece; and 

14 cause the database to be rendered into one or more images from one or more 

15 different viewpoints; 

16 cause the one or more images to be displayed on the one or more display devices; 

17 and 

18 construct virtual objects within the virtual world using a second point hierarchy 

19 and a data flow network for controlling the motion of nodes of the virtual 

20 objects by: 

21 attaching each virtual object node hierarchically with at least one other virtual 

22 object node to form the second point hierarchy, wherein each of the virtual 

23 object nodes has a position and an orientation, and 

24 building the data flow network as an interconnection of input units, function 

25 units, and output units, wherein said input units receive data from sensors 

26 and output the received data to at least one of said function units, wherein 

27 each of said function units includes at least one input and at least one 

28 output, each function unit generating a value for the at least one output 

29 based on at least one of data received from at least one of the input units 

30 and data received from an output of at least one other of said function 

31 units, and wherein the output units generate the position and the 

32 orientation of a portion of the nodes to the virtual objects. 
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